Third party action triggers

ABSTRACT

Methods, systems, and apparatus, for providing notifications based on third party data. In an aspect, a method includes receiving first instructions that cause a user device to render a resource that includes a first parameter value of a first parameter and display the resource and generating second instructions that cause the user device to display an action trigger prompt that is rendered with the resource and in an initial state, and wherein the action trigger prompt is rendered to augment the resource with additional content not provided by rendering of the resource according to the first instructions.

CLAIM OF PRIORITY

This application is a continuation of U.S. application Ser. No.15/197,301, filed Jun. 29, 2016, which application is a continuation ofU.S. application Ser. No. 14/752,369, now U.S. Pat. No. 9,408,182, filedJun. 26, 2015, which claims the benefit of U.S. Provisional ApplicationSer. No. 62/167,726, filed on May 28, 2015. The entire contents of eachof the foregoing applications are incorporated herein by reference.

BACKGROUND

The advent of cloud based services, search engines, mobile applications,and location-aware devices have drastically expanded the utility ofmobile user devices over the last decade. Many such user devices nowprovide context-aware services and applications in addition to voice anddata access. Furthermore, with the recent advances in processingsystems, many previously disparate applications and services are nowable to work together to provide enhanced capabilities and userexperiences.

Many of these application services available to users are instantiatedby use of command inputs. One such service is creating action triggers.For example, a user may speak (or type) the input [remind me to buy milkthis evening] into a smartphone, and the smartphone, using a commandparsing application (or, alternatively, communicating with a commandparsing service) will invoke an action trigger process that may solicitadditional information from the user. Such information may be a time, ifthe user desires for the action trigger to be performed at a certaintime, and/or a location, if the user desires for the action trigger tobe performed when the user arrives at the location. While the setting ofsuch action triggers are very useful in a relatively fluid userexperience, the users often forget to do the things they wanted to dobecause they cannot setup action triggers that are based on the contextthat they need to be in to be able to complete the task at hand.

SUMMARY

This specification relates to action triggers, which are associations ofuser defined events and actions, that are generated by use of an actiontrigger prompt rendered with a resource. In an example implementation,application pages and web-based resources include an action triggerprompt though which a user may set up a notification that is specific tothe user. The notification is subject to parameter value criteria set bythe user. The publishers of the applications and web-based resources,which are generally referred to herein as “third parties,” may thenprovide a notification to the user once the criteria for thenotification are met. Alternatively, the third parties may provide datato a system that, independent of the third party, manages the actiontriggers for multiple different third parties. The third parties mayprovide the data to the system automatically, or the data may berequested, searched, or otherwise gathered by the system.

In additional implementations, the action trigger prompts and the actiontriggers created by use of the action trigger prompts may be implementedentirely independent of third parties. For example, the systemsdescribed in this specification may be configured to identify theparameters rendered on a resource, and for at least one of the parametervalues displayed on the resource, render an action trigger prompt withthe rendered resource. This frees the third parties of back-end systemdevelopment and resource layout design and coding efforts. However, thedevelopers of the third-party content will still benefit for increaseduser engagement, as described in more detail below.

The action that can be taken in response to an event occurring can beuser defined. In some implementations, the action, when taken, may ormay not enable the user to take subsequent actions. In the case of theformer, for example, the action may be a notification that an event hasoccurred, or, alternatively, the action may be performed by the userdevice, or by some other device or party, without notification to theuser. In the case of the latter, for example, the action may be aconfirmation prompt to take a subsequent action. For example, the usermay have set an action trigger with an event of Company X stock fallingbelow $620 a share, and may have set the action to be a presentation ofa confirmation prompt to purchase 10 shares of Company X stock when theevent occurs. The user may then accept or reject the confirmation whenpresented with the confirmation prompt when the action occurs. Shouldthe user accept the confirmation, a subsequent action that results inthe purchase of the stock can be performed (or, alternatively, the usermay be presented with a resource through which the user may purchase thestock).

In some implementations, the notification is provided to a softwareassistant for the user. For example, the third party may have aproprietary data source which when a user defined event is met the thirdparty system notifies the personal assistant application. Alternatively,the personal assistant application may monitor the data source, orcontent published by a resource, and detects the occurrence of theevent. The software assistant then determines the importance of thisnotification and surfaces it to the user in the best determined way,e.g., according to one or more conditions that are either automaticallydetermined by the assistant or set by the user.

The notification may be informative only, or may alternatively beinteractive. In the latter case, if the user interacts with thenotification, the user may be linked back to the initial application orweb page from which the user set the notification, or some otherresource that permits the user to execute an action related to thenotification. This allows for a very efficient engagement model with theuser and that brings the user to the third party service at a desiredtime.

Implementations of the subject matter described below allows for anintuitive, more accurate, and more tailorable user experience whencreating action triggers. The creation of an action trigger that iscontextually relevant to content that the user is viewing allows theuser to set a reminder or action and be alerted of events relevant tothat content. This provides a very fluid and intuitive interaction modelfor the user that makes it easier for the user to set up an alert oraction.

For example, a user may be viewing a resource through which tickets forconcerts may be purchased. The user may search for a particular artistand may learn that the artist is not currently on tour. By use of theaction trigger, however, the user may set up a reminder to be notifiedwhen new tour dates are published for the artist, or to evenautomatically purchase tickets for a particular tour venue when thetickets become available This reduces the necessity for a user to keep aparticular schedule or check the resource frequently, and can facilitatenotifying the user of user defined events or performing user definedactions when the user defined event has occurred. The notifications maybe made in real-time to better equip the user to respond in a timely andefficient manner. Additionally, in some implementations, the user canselect for the user defined action to be completed once the user definedevent has been determined to occur.

A variety of notification types may be used, including a mobileoperating system (OS) push notification, text message, email, a web ordigital assistant alert, or other form of notification. As describedabove, in the case of a digital assistant alert, the digital assistantmay determine the proper time, place, and method to notify the user ortake action on the user's behalf. For example, if the user is in abusiness meeting, the digital assistant may wait to notify the useruntil the user is out of the meeting, or after the user's work day iscompleted.

The details of one or more embodiments of the subject matter describedin this specification are set forth in the accompanying drawings and thedescription below. Other features, aspects, and advantages of thesubject matter will become apparent from the description, the drawings,and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an example environment in which commandinputs are processed for action triggers.

FIG. 2A is an illustration of an interface of a resource in a portion ofa web page.

FIG. 2B is an illustration of an interface of a resource in a portion ofa web page after selecting the action trigger prompt.

FIG. 2C is an illustration of an interface of a resource in a portion ofa web page in selecting the second parameter value in the action triggermenu.

FIG. 2D is an illustration of an interface at a user.

FIG. 2E is a flow diagram of an example process for creating an actiontrigger and performing an action based on the action trigger.

FIG. 2F is an illustration of an action selection menu.

FIG. 3 is a flow diagram of another example process for creating anaction trigger.

FIG. 4A is an illustration of a user interface at a user device in whichan action trigger prompt is provided along with a resource.

FIG. 4B is an illustration of a user interface where an action triggermenu is provided.

FIG. 4C is an illustration of a user interface providing a notificationthat the value of the first parameter is the parameter value of valueinput.

FIG. 5 is a block diagram of an example mobile computing device.

Like reference numbers and designations in the various drawings indicatelike elements.

DETAILED DESCRIPTION

An action trigger processing system facilitates action triggersincluding user defined events. In operation, a resource, e.g., a webpage resource or a particular application page in an application,includes instructions that cause the user device to display an actiontrigger prompt that is rendered in an initial state. The resource isprovided to the user device, and the resource and action trigger promptare rendered at the user device. An action trigger menu that includes aninput for a user defined event may be provided if the action triggerprompt is selected at the user device. The action trigger menu may alsodefine one or more actions that can be taken when the event occurs. Theassociation of the action and the events is referred to as an “actiontrigger,” and the action trigger may be stored in an action trigger datastore.

The action trigger processing system obtains updates and informationassociated with resources from event data (e.g., by searching orreceiving data of the resources), and when the action trigger processingsystem determines that the user defined event of the action trigger hasoccurred, the action trigger processor may perform the action when theevent occurs. The action may be an action that is performedautomatically with or without a notification to the user, or may be theproviding of a notification to the user, or may be the providing of aconfirmation prompt to take a subsequent action. In someimplementations, the user defined action for the action trigger may beinitiated when the action trigger processing system determines the userdefined event has occurred. In some implementations, an automaticoperation may be completed after the user defined event has occurred(e.g., automatic ticket purchase when concert tickets go on sale).Further, in some implementations the action trigger processing systemmay receive a selection by the user of one or more trigger conditionsthat indicate a condition to be satisfied in determining that the userdefined event has occurred.

In some implementations, code for the action trigger prompt may beincluded in the web page resource or application page. In otherimplementations, the code may be inserted into the web page resource orapplication page. In still other implementations the action triggerprompt may be rendered as an overlay to the web page resource orapplication page.

The action trigger processing system can be implemented in the userdevice, or in a computer system separate from user device, such as aserver system. In the case of the latter, the server system receivesinput from the user device and sends data to the user device forprocessing. These features and additional features are described in moredetail below.

FIG. 1 is a block diagram of an environment 100 in which command inputsare processed for action triggers including user defined events. Acomputer network 102, such as the Internet, or a combination thereof,provides for data communication between electronic devices and systems.The computer network 102 may also include, or be in data communicationwith, one or more wireless networks 103 by means of one or moregateways.

A resource publisher 104 includes one or more resources 105. There maybe one or more resource publisher 104. The resource publisher 104 may bea website publisher, an application publisher, among others. Inimplementations where the resource publisher 104 is a website publisher,the one or more resources 105 are associated with a domain and hosted byone or more servers in one or more locations. Generally, a resourcepublisher website is a collection of web pages formatted in hypertextmarkup language (HTML) that can contain text, images, multimediacontent, and programming elements. Each website is maintained by acontent publisher, which is an entity that controls, manages and/or ownsthe website. A web page resource is any data that can be provided by apublisher resource 104 over the network 102 and that has a resourceaddress, e.g., a uniform resource locator (URL). Web resources may beHTML pages, images files, video files, audio files, and feed sources, toname just a few. The resources may include embedded information, e.g.,meta information and hyperlinks, and/or embedded instructions, e.g.,client-side scripts. More generally, a “resource” is anythingidentifiable over a network.

A resource, as used in this specification, may also be a particularapplication page of a stand-alone application that is run separate froma web browser. For example, a ticket selling agency may publish anapplication that sells tickets to various events. The application mayhave various pages, such as a page for musical artists, another page forsporting events, and another page for plays, for example.

User device 106 is an electronic device that is under the control of auser and is capable of requesting and receiving resources over thenetwork 102, also capable of performing other actions. Example userdevices 106 include personal computers, mobile communication devices,wearable devices, and other devices that can send and receive data overthe network 102. In the example of FIG. 1, the user device 106 is asmartphone. An example smartphone is described with reference to FIG. 5below. The user device 106 may communicate over the networks 102 and 103by means of wired and wireless connections with the networks 102 and103, respectively. The user device 106 may also be a device thataccesses the network 102 by means of an intermediate device and a shortrange wireless transceiver, such as a WiFi, Bluetooth, etc. As describedwith reference to FIG. 5, a user device may be able to perform a set ofdevice actions for various programs and capabilities.

The user device 106 is associated with a user account, such as anaccount hosted by a cloud service provider 112 that provides multipleservices. These services may include search (web, intranet, documents,and applications, among others), web mail, calendar, social networking,messaging, documents storage and editing, an electronic assistantservice, etc. The account data 114 may store data specific to theaccount of the user device 106. Further, although only one user device106 is shown in FIG. 1, a plurality of user devices 106 may be included.

In some implementations, the user may invoke actions and events thatcause the actions to be performed by means of an action trigger promptprovided with resources. The generation of action triggers, which are anassociation of user defined events and user defined actions, and theprocessing of such items are described in more detail below. An actiontrigger processing system 120 receives the action triggers, either fromthe third party publishers or user devices, and stores the actiontriggers in an action trigger data store 126. While the action triggerprocessing system 120 is shown as a separate entity in FIG. 1, theaction trigger processing system 120 can be implemented in the cloudservice provider 112, or alternatively as a separate system by eachthird party. In certain embodiments, action trigger processing system120, or subparts thereof, may form a portion of or be stored on userdevice 106.

Action triggers that are stored in the action trigger data store are theassociation of actions and events. The action trigger processor 124accesses event data 128 to determine which events are met, and whetherto perform the actions associated with the events that are determined tobe met. As used herein, event data 128 is any data that can be used todetermine whether an event has occurred. The event data may be data thatis preprocessed and sent to the action trigger processor, or may beother data that can be searched by or on behalf of the action triggerprocessor 124. For example, the event data 128 may be data that isprovided to the action trigger processing system 120 by the thirdparties, or may be data that is requested by the action triggerprocessing system periodically, or may even be an indexed corpus ofresources, knowledge graph, or any other data source that may besearched for parameter values to determine whether an event hasoccurred.

The action trigger processor 124 will store the action trigger in anaction trigger data store 126. There may be a plurality of actiontriggers AT1, AT2, . . . ATn stored in action trigger data 126. A groupof action triggers (e.g., those created or previously implemented by theuser) may be viewed at the user device 106, and the user may makechanges (e.g., add, delete, modify) at the user device 106 to the actiontriggers at any time. Each of the plurality of action triggers may haveone or more user defined events E1, E2, . . . En associated with anaction A. Upon the occurrence of a user defined event, the actiontrigger processor 124 may cause the action to be performed. The actionmay be one or more of: the presentation of a notification that the eventoccurred; an action that is performed partly or fully automatically andseparate from the user device; or the presentation of a prompt toperform another action. For example, based on the example above, theuser selects or specifies a user defined action, such as “buy tickets,”to take place when the user defined event occurs.

Additionally, in some implementations, a user may provide triggerconditions Tc1, Tc2, . . . Tcn associated with the one or more actiontriggers. Generally, the trigger conditions are additional conditions orevents that are not available in the action trigger prompt menu but thata user can associate with an action. A trigger condition specifies acondition to be satisfied in determining that the user defined eventindicated in the action trigger has occurred, or that may be required tobe satisfied before the user is notified of the events being met. Forexample, suppose a user has set an action trigger to be notified whentickets become available for an artist.

The user may also specify that notifications should be provided onlyduring appropriate times during which the user is able to review andrespond to the notifications. For example, the user may also set acondition that the user is not to be notified of tickets comingavailable based on the user's availability according to the user'scalendar. Thus, the presentation of the notification may be delayed whenthe user's calendar indicates the user is currently engaged in anappointment, e.g., if the user is in a meeting as scheduled on theuser's work calendar.

Trigger conditions may be one or more time period condition, locationarea condition, or person proximity condition, among others. A timeperiod condition may be a date, a date range, a time of day, or a timeof day range, among others. For example, time period condition may be adefault or user set time range (e.g., 1 PM-5 PM) on a particularSaturday (e.g., the next Saturday), every Saturday, selected Saturdays,or a pattern of Saturdays (e.g., the first Saturday of every month).

A location area condition may be an area around a particular location(e.g., house address) or type of location (e.g., grocery store, airport,hospital) that the user device 106 is to be within or near for the userdefined event to be met. For example, the location area condition may be“At Home,” which may be defined by the user on the user device 106.Additionally, “near” can be a particular distance from (e.g., feet ormiles) or amount of time away by different modes of transportation(e.g., by car, public transportation, walking) from the identifiedlocation. Particular location types may be specified by the user, or maybe inferred. For example, in the case of the latter, the location atwhich a user is most often located during the evening and throughmid-morning on weekdays may be inferred as “Home,” and the location thatthe user is most often located during late morning and afternoons onweekdays, if different from the inferred “Home” location, may beinferred as “Work.”

Additionally, a trigger condition may be a person proximity condition. Aperson proximity condition may be met if the user device 106 of the useris within a threshold distance from an identified user device of aparticular person or group.

In some implementations, when creating an action trigger, a user definedevent may be created based on the content of the resources 105, and aparameter may be selected or input at the user device 106. An example ofthis is illustrated with reference to FIGS. 2A-2D. The flow throughFIGS. 2A-2D is described with reference to FIG. 2E, which is a flowdiagram of an example process 250 for creating an action trigger andperforming an action based on the action trigger. The process 250 may beimplemented in one or more computers that are used to realize the actiontrigger processing system 120.

The process 250 receives data responsive to a selection of an actiontrigger prompt at a user device (252). An action trigger prompt is anelement, such as a user interface element in the form of a button,pull-down menu, audible cue, etc. that provides a user with the optionto initiate setting an action trigger. The action trigger prompt isrendered or provided at the user device with a resource and displayedwith the resource, and is rendered or provided in an initial state. Uponselection or activation of the action trigger prompt the user devicedisplays an action trigger menu that facilitates the input of multipleparameter values for an event. This selection and menu for the actiontrigger prompt is shown with reference to FIGS. 2A-2C. In particular,FIG. 2A is an illustration of an interface 202 a of a resource in aportion of a web page. The web page may include a list and chart ofmultiple stocks. The web page 202 a includes an action trigger prompt204 in an initial state (e.g., prior to selection) and a first parametervalue 206 of a first parameter. In this case, the first parameter is thestock price of the stock “XCOMPANY” and the first parameter value is$624.36.

FIG. 2B is an illustration of an interface 202 b of a resource in aportion of a web page after selecting the action trigger prompt. Uponselection (e.g., by the pointer) of the action trigger prompt, an actiontrigger menu is provided that includes an input for a user definedevent. The selection generates an action trigger menu that facilitatesthe input of multiple different possible parameter values for an event.

For example, the user may desire an action trigger for a certain stockprice going above or below a certain price, or a certain stock pricestaying at a value for a period of time, or a certain stock price at aparticular time, among others. In the current example, action trigger isrelated to a certain stock going above a certain price. The user mayselect a stock ticker 208, which in the current implementation is theparticular stock of XCOMPANY, which may be changed by the user. Also, asecond parameter value 210 may be selected by the user. In someimplementations, a default value may be provided, which could be basedon the user history, selections made by other users, or default values,among others. In the current example, a default value of $630 isprovided.

In certain embodiments, one or more parameters are automaticallyselected based on context, such as the content of the web page withwhich the action trigger prompt 204 is associated or on which it isdisplayed. For example, when viewing the web page 202 a showing stockprices for Company X, selection of action trigger prompt 204 displayedwith the parameter value 206 automatically provides the ticker symbolXCOMPANY for Company X as a parameter. While this parameter may bechanged by the user, the context of the display, user actions,environment, and so on may be used to automatically (and at leastinitially) set one or more parameters. It is contemplated herein thatthis functionality can be provided in any or all embodiments in whichthe action trigger prompt is included in the web page resource orapplication page, inserted into the web page resource or applicationpage, and/or is rendered as an overlay to the web page resource orapplication page.

In the case of an overlay, various processes can be used to detectparameters and generate an overlay. For example, a document object modelof a resource may be processed to identify predefined parameter typesand values, e.g., stock symbols and prices, and an overlay may berendered by use of iFrame or any other process by which content of adisplayed resource may be augmented with additional content. Forapplication pages, a textview object may be extracted from displayeddata and processed to identify the predefined parameter types andvalues, and an overlay can be generated by a separate process thatdisplays additional content in an active region that defines only theaction trigger in the viewport. The overlay is generated by use ofinstructions provided to the user device and that are separate from theinstructions that are used to render the resource.

In variations of this implementation, image processing can be used toidentify textual content of an image and provide an action menu forcertain textual content. In particular, the action trigger processingsystem 120 may receive data describing textual content recognized for animage taken by a user device and displayed on the user device. The userdevice may perform the textual recognition and generate data describingthe textual content and the location of the textual content in the imageand send the data to the action trigger processing system 120.Alternatively, the user device may send the image to another system thatprocesses the image that provides the textual data and location data tothe action trigger processing system 120. A variety of text recognitionprocessing techniques can be used.

For example, suppose a user takes a photograph of a tour poster for anartist. The action trigger menu will receive textual data describingcontent of the tour poster, and may generate an action menu near thetext describing the tour. The action menu may include notificationsactions to inform the user of various tour dates when the tour isannounced, purchase actions to purchase tickets when the tickets go onsale, and so on. The action menu is rendered over the image displayed onthe user device.

To generate the instructions for the overlay, the system 120, or,alternatively the user device 106, may process the rendered resource asdescribed above. Upon the detection of certain parameter values, e.g.,parameter values for predefined parameters, such as stock symbols,prices, dates, and the like, the system 120, or the user device 106,generates the overlay instructions for processing by the user device.

In some implementations, certain resources may be identified in advanceof rendering and a list of the identified resources may be provided tothe system 120 or user device 106. Thereafter, when one of theidentified resources is rendered at the user device 106, the user devicemay notify the system 120 and the system 120 may generate the overlayinstructions. Alternatively, the user device may generate the overlayinstructions.

FIG. 2C is an illustration of an interface 202 c of a resource in aportion of a web page, illustrating selecting the second parameter value210 in the action trigger menu. In the current implementation, uponselection (e.g., by the pointer) of the second parameter value 210, asecond parameter value option list 212 may be provided. The secondparameter value option list may include user-selectable options for thesecond parameter value. The list of options may be based on the userhistory (e.g., selected a particular stock price or certain percentageabove the current stock price in the past), context (e.g., other databeing shown in interface 202 c), other selections provided by otherusers, or a default set of values, among others. The user may select oneof the second parameter value options from the option list 212, orselect to “Set Amount,” where the user may provide the second parametervalue option by inputting a value. In the current example, the user hascreated a user defined event with a first parameter of “XCOMPANY stock,”and a second parameter value of $630.

Upon selecting 630, an action trigger is created. The action trigger inthis case is a notification that Company X stock is above $630 a share,and the event that triggers the notification is the value of Company Xstock exceeding $630 a share. The action trigger is thus an associationof the user defined event and the action to be taken in response to theoccurrence of the user defined event. It will be appreciated thatadditional fields for user-selectable parameters may be provided. Forexample, a field may be provided permitting a user to specify whataction from multiple different actions to take when Computer Co. stockis above $630 per share, such as place a sell order for [x] shares, orsend an email to the user's stockbroker, and so on.

In some implementations, actions can be automatically executed by athird party system by use of an API to the action trigger processingsystem 120 or personal assistant service. The action trigger processingsystem 120 or personal assistant service may store credentials (e.g.,usernames, passwords, account information, etc.). These third partyactions may also make use of an electronic wallet. Such third party APIlevel integrations allow the action trigger processing system 120 orpersonal assistant service to complete financial transactions, e.g.,stock trade, ticket purchases, etc., on behalf of the user.

The process 250 stores the data specifying the user defined event andthe action associated with the user defined event as an action triggerin the action trigger data store (254). Thereafter, the process 250 willperiodically monitor for the occurrence of the user defined event (256),and in response to the occurrence of the user defined event, cause theaction associated with the user defined event to be performed (258).

Continuing with the example, the action trigger processor 124 willmonitor the stock price of Company X. The monitoring can be done in avariety of ways. For example, the action trigger processor 124 mayrequest to be notified by a third party service when the stock exceedsthe value of $630. Alternatively, the action trigger processor 124 mayperiodically search a web corpus or other data source to determinewhether the event has occurred.

FIG. 2D is an illustration of an interface 202 d at a user device 106when there has been an occurrence of the user defined event provided inFIG. 2C. In the current implementation, after creating the actiontrigger and storing the association of the action and the event in theaction data store 126, the action trigger processing system 120 mayreceive updates and/or check data sources associated with the userdefined event and action trigger to determine if the user defined eventhas occurred. In this example, the action trigger processor 126 willdetermine whether the value of the first parameter, the stock price,meets the second parameter value of $630.

In the current example, the action trigger processing system 120 hasmade the determination that there has been an occurrence of the userdefined event, and a notification 214 is provided to and/or by the userdevice 106. Additionally, in some implementations, the notification maybe provided by the action trigger processor 124, and the user of theuser device 106 may access information related to the user definedaction. The notifications the user receives may be notificationsprovided on a mobile device (e.g., Android notifications, In-appnotifications, etc.), or notifications on a desktop (e.g., browsernotifications, OS notifications, etc.). Within the notification, a linkor button to perform an operation (or user defined action) associatedwith the action trigger may be provided.

In the current example, the action specified is a notification action.In some implementations, the action trigger prompt 204 may also includean action selection menu that includes a list of selectable actions. Theactions may include, for example, actions that can be taken on behalf ofthe user with or without notification. The actions may be contextual inthat the actions available are dependent on the type of content shown inthe resource. The system 120 may have data pre-associating particularactions with particular content types. For example, in addition to the“Let me know” selection shown in FIGS. 2A-2C, other actions may be“Purchase stock,” “Sell stock,” etc., by means of an action selectionmenu 205 as shown in FIG. 2F. Similarly, if the resource was a ticketpurchasing resource, actions may include notifications when tickets goon sale, notifications when an artist announces tour dates, and thepurchasing of tickets.

By way of a further example, suppose FIG. 2A depicts not a web page butan image of a stock ticker report taken by the user device. Using theimage processing techniques described above, the same action triggerprompt 204 can be generated and overlaid on the image as shown.

In some implementations, selecting the notification may cause the userdevice to display the resource from which the action trigger was set.Alternatively, if the action was, for example, to purchase stock,selection of the notification may cause the user device to display anenvironment from which the user may purchase the stock or viewconfirmation that the stock was purchased.

In some implementations, a notification may be subject to additionalpresentation criteria. The presentation criteria may be selected, forexample, by a software assistant, or alternatively selected by a user.In the case of the former, with respect to notifications, a softwareassistant may determine from a user's schedule data whether to present anotification to a user, i.e., a notification may only be presented ifthe user is not in a meeting, etc. In some implementations, thevolatility of the underlying data related to the event that caused thenotification may be a factor considered by the software assistant. Forexample, the software assistant may override a user's meeting status andpresent a notification while a user is in a meeting if, for example, thenotification relates to an option to purchase stock and a pricespecified by the user. Because the volatility of the stock price maychange rapidly, the user may benefit from an immediate notification.Conversely, notifications for events for which the underlying datachanges less frequently may be delayed and presented to the user at amore convenient time for the user.

By way of another example, when a user selects the action trigger prompton the user device 106, the action trigger menu may be provided thatincludes an input for a user defined event. When a resource has multipledifferent parameters available, the user may be able to select a firstparameter. Example parameters may be topics, titles, categories, orother information provided within the content of the resource 105. Forexample, if the user is viewing a resource related to tickets for ArtistA's concerts and the user selects the request prompt provided on theresource, parameters associated with the resource may be concerttickets, locations of concerts, dates of a concert tour, or any otherinformation provided within the content of resource 105. Within therequest prompt, the user may input or select a parameter for the userdefined event.

The action trigger processor 124 may determine a value for the parameterbased on the content provided in the resource. For example, if the userselected concert tickets as the first parameter, then the value for thatparameter may indicate the status of the concert tickets, which may be“Not available,” or some other indication, from the content of theresource, that the tickets are not available (e.g., sold out or not yeton sale).

After selecting or inputting a parameter, the user may input a parametervalue for the indicated parameter to define the user defined event ofthe action trigger. The user provided parameter value may be differentfrom the parameter value displayed in the resource. For example, theuser may create a user defined event to be notified when the concerttickets go on sale, or are otherwise available. Thus, the parametervalue specified by the user would be “available.”

Additionally, in some implementations, an action trigger may includemultiple parameters. For example, based on the example above, the actiontrigger may include a second parameter related to ticket sections orticket price ranges for Artist A's concerts. The second parameter may berelated to, for example, Artist A's concert ticket prices, and theparameter value provided by the user may be, for example, “less than$40.” In the current example, the user defined event would not be metuntil the user provided parameter values for the first parameter and thesecond parameter occur, e.g., tickets are available and less than $40.

The action trigger is then created and stored in the action trigger data126. After the action trigger is created, the action trigger processormonitors for the occurrence of the event(s) associated with the action.Action trigger processor 124 may continuously or periodically (e.g.,poll the event data 128 at regular or irregular intervals) search thecontent, or register for a notification from a third party of datasatisfying the event, etc.

As described above, event data may include resources 105. Some contentof resources 105 may be information that is generally available andaccessible from the resource publisher 104 (e.g., application publisheror website publisher). These resources 105 may be, for example, relatedto weather, news, entertainment, or any other type of resource that doesnot require a subscription or authentication to see the information andcontent of the resource 105.

Further, some content of resource 105 may be information that is notpersonalized to a particular user, but the content nonetheless mayrequire the user to sign in, or otherwise provide authorization, foraccess to the content of the resource 105. These resources 105 may be,for example, related to newspapers, journals, or blogs that require asubscription or other type of verification or login from the user inorder to access all of the content of the resource.

Moreover, some content of resource 105 may be information that ispersonalized, and possibly private, to a particular user. Theseresources may be, for example, related to bank account information orsocial media websites or applications. These resources 105 may beproprietary to the resource publisher 104 and can also require login orauthentication information from the user to access the personalizedinformation and content of the user. The resource publisher 104 may, insome implementations, select or enable individual users to decide ifthey want their personalized content associated with the resources 105to be accessible to the action trigger processor 124.

Content of resources 105 that require authorization may not be includedin the event data 128 unless the action trigger processing system 120has access to the authorization information for the user of the userdevice 106. The authentication information for a particular user of theaction trigger processing system 120 may be stored and accessed indifferent ways, and different amounts of authentication information maybe stored for each user depending on, for example, selections indicatedby the user (e.g., a user may select for the action trigger processingsystem 120 to not store or access any authentication and logininformation for that particular user). For example, a user may provideauthentication information, where required, to the action triggerprocessing system 120 for the resources 105 associated with the resourcepublishers 104 they would like to be included in the event data 128(i.e., the data that is accessible to the action trigger processor 124).

Additionally, in some implementations, the user may enable the actiontrigger processing system 120 to access and keep track of authenticationinformation for resource publishers 104 that the user device 106 of theuser has accessed (e.g., a user may enable the action trigger processor124 to store the authentication information that accesses resourcesrelated to bank account information of the user at a publisher resource104 of a particular bank).

Moreover, a user history related to previous actions performed by theuser on the user device 106 may be included in action trigger data 126.For example, previous action triggers, user defined events, and userdefined actions may be stored in the action trigger data 126 to enabledefault, favorite, or common action triggers, user defined events,and/or user defined actions, for the particular user, a group of users,and/or a subset of users, to be selected by the user. Additionally, acontext of a user may change or adjust the suggested or favorite actiontriggers, user defined events, and/or user defined actions. The time ofday, week, month, location of the user, frequency and regularity ofsetting up particular action triggers, among others may factor in to thecontext of the user. However, in the example above, the user can changethe parameters and parameter values that are initially suggested orprovided to the user. Additionally, in some implementations, a usercalendar on a user device 106 and user interests provided, implicitly(e.g., based on a user history) or explicitly, to the action triggerprocessing system 120 may factor in to determining suggested actiontriggers, user defined events, user defined actions, parameters andparameter values. Moreover, the order or ranking of suggested actiontriggers provided (e.g., as the user is inputting an action trigger) mayalso adjust and change based on context, prior action triggers, or otherinformation.

FIG. 3 is a flow diagram of an example process 300 for creating anaction trigger from a resource 105 of a resource publisher 104. Theprocess 300 can, for example, be implemented by the action triggerprocessor 124. In some implementations, the operations of the exampleprocess 300 can be implemented as instructions stored on anon-transitory computer readable medium, where the instructions cause adata processing apparatus to perform operations of the example process300.

A resource 105 that includes a first parameter value of a firstparameter and instructions that cause the user device 106 to display anaction trigger prompt that is rendered in an initial state is provided(302). Each resource 105 may include content and the action triggerprocessor 124 may determine one or more parameters from the contentprovided in the resource 105. For example, the first parameter may betopics, titles, categories, or other information provided within thecontent of the resource 105. The first parameter value may indicateinformation provided in the content of the resource 105 related to thefirst parameter. For example, the first parameter value may indicateavailability, cost, or a weather condition, among others.

The action trigger prompt may be displayed along with or within theresource 105. The action request prompt may be included from theresource publisher 104 when the resource 105 is provided to the userdevice 106, or in other implementations, the action trigger processingsystem 120 may provide the action request prompt to the user device 106,where the action request prompt is provided with the resource 105 oroverlaid over the resource 105. If the action request prompt is includedfrom the resource publisher 104, the action trigger processing system120 may provide the required scripting to the resource publishers 104that include the action triggers (e.g., via API 107).

Instructions are provided that render the resource 105 at the userdevice 106 (304). The resource 105 may then be rendered at the userdevice along with the action trigger prompt. A selection of the actiontrigger prompt at the user device 106 is received (306).

Instructions that cause the user device 106 to display an action triggermenu that includes an input for a user defined event are provided (308).After selection of the action trigger prompt, the action trigger menumay be provided where the user, via the user device 106, can select orinput a user defined event to be notified of when the event occurs, orin some implementations, a user defined action may be initiated orcompleted when the particular user defined event occurs.

Input of the first parameter of the user defined event in the actiontrigger menu is received, where the resource 105 includes a firstparameter value for the first parameter (310). The user defined eventmay include a first parameter that is input or selected by the user atthe user device 106, as previously described. The first parameterincludes a first parameter value, based on the content provided in theresource 105. For example, the first parameter value may indicate astatus, availability, cost, or a weather condition, among others. In anexample of a particular stock quote shown at a resource 105 (e.g., a webpage or mobile application) including information of a particular stock,a first parameter may be, for example, the stock price, earnings,opening and closing price, or other type of information that may beobtained from the resource 105.

For example, if the first parameter is the stock price, and the firstparameter value may be the current price or what is listed as thecurrent price at the resource 105, for example $600 per share. Likewise,in a resource 105 that includes concert tickets for a particular artist,the first parameter may be ticket availability (e.g., a new tour for theartist), ticket price, seat location, concert location, or other type ofinformation that may be obtained from the resource 105. In the currentexample, the first parameter is ticket availability, and the firstparameter may indicate that concert tickets for the particular artistare unavailable.

A value input as a second parameter value for the first parameter of theuser defined event is received, wherein the second parameter value onlyspecifies a parameter value that is different from the first parametervalue (312). The user defined event may also include a second parametervalue that is input or selected by the user at the user device 106. Thesecond parameter value may indicate the event related to the identifiedfirst parameter that the user would like to be notified of, or a userdefined action to take place when the second parameter value is reachedby the first parameter, based on the content of the resource 105.

The action trigger processor 124 may search the event data 128 todetermine a parameter value of the identified parameter. Based on thestock price example above, the second parameter value of the stock pricemay be provided by the user, and may be, for example, $630 per share.When the action trigger processor 124 determines from the data providedin the resource 105 and/or the event data 128 that the value of thefirst parameter has met the value of the second parameter value, then anotification of such may be provided to the user device 106 (e.g., stockof Company X has reached $630) or a user defined action may be set totake place (e.g., the action trigger system 120 may be directed to sellthe stock). Likewise, with respect to the concert ticket example above,a second parameter value of ticket availability may be, for example, anew tour (i.e., tickets for sale). When the action trigger processor 124determines from the data provided in the resource 105 and/or the eventdata 128 that the value of the first parameter (ticket availability) hasmet the value of the second parameter value (tickets for sale), anotification of such may be provided to the user device 106 (e.g.,tickets for Artist A are now on sale) or a user defined action may beset to take place (e.g., the action trigger system 120 may be directedto purchase tickets).

For example, FIG. 4A is an illustration of a user interface 402 a at auser device 400 in which an action trigger prompt is provided along witha resource 105. Within resource 105, which in the current example may bean event ticket selection and purchase web site or application, a searchinput 404 may be provided. In the current example, “Artist A” has beenprovided in the search input 404, and the resource 105 has indicatedthere are not currently any events for “Artist A.” Also, a locationidentifier 405 is provided in the resource 105, which may be a currentlocation of a user device 106 of the user or a location the user hasprovided or input. Additionally, an action trigger prompt 406 isprovided in the resource 105 to enable the user of the user device 106to create an action trigger.

In FIG. 4B, an illustration of a user interface 402 b provided where theaction trigger prompt 406 has been selected, and an action trigger menu408 is provided. In the current example, based on the content providedin resource 105, the action trigger processor 124 may provide asuggested user defined event and the user may be able to input orprovide a different selection for the user defined event (e.g., by useof the arrows in action trigger menu 408). The suggested user definedevent may be based on the information of action trigger data 126 and/orevent data 128. In the current example, a suggested user defined eventof “When Artist A has a new tour” is provided. The first parameter maybe determined to be related to Artist A's tours, and based on thelocation content provided in resource 105, the first parameter may berelated to Artist A's tours in the “SF Bay Area, Calif.”

The first parameter value, in the current example, may indicate that theconcert and tour information for the first parameter is not available.In some implementations, the first parameter value may be input orprovided by the user of the user device. At value input 410, the usermay provide a parameter value of the first parameter to complete adefinition of the user defined event, and when the first parameter atresource 105 has a value of the user provided parameter value, then theuser defined event may be met. In the current example, the user providedparameter value is related to Artist A's “tour,” and specifically a newtour. In some implementations, based on the content of resource 105 andthe information of action trigger data 126 and/or event data 128, whichmay include, for example, interests and a user history, of the user ofuser device 106, a suggested or hierarchy of user provided values toinclude in value input 410 may be provided.

In FIG. 4C, an illustration of a user interface 402 c is providedshowing a notification that the value of the first parameter has met theparameter value of value input 410, or a second parameter value. Aspreviously described, action trigger processor 124 obtains updates andstatus information associated with resource 105 from event data 128, andwhen the action trigger processing system 120 determines the userdefined event has occurred, the action trigger processor may provide anotification that the value of the first parameter is the secondparameter value to the user device, or in some implementations, a userdefined action may be complete when the action trigger processing system120 determines the user defined event has occurred. In the currentexample, FIG. 4C indicates that the user defined event has occurred, anda user defined event notification 412 is provided to the user device 106of the user. In some implementations, a user may select the user definedevent notification 412 and the user may be provided with informationassociated with the user defined event, and in the current example, theuser may be directed to the resource 105 to see information related toArtist A's tour and see ticket purchase selections.

In some implementations, the user of the user device 106 may select orprovide a user defined action to occur once the action triggerprocessing system 120 determines there has been an occurrence of theuser defined event. For example, based on the example provided in FIGS.4A-4C, the user may provide a user defined action to the action triggerprocessing system 120, via the user device 106, to purchase tickets whenresource 105 indicates that Artist A has a new tour (and tickets areavailable). Additionally, the user defined action may specify, forexample, a number of tickets, a date or date range, a price range, and asection in which to purchase the tickets. Additionally, a notificationmay also be provided to indicate that the user defined action has beenperformed (and a notification may also be included to indicate the userdefined event has occurred before or while in the process of performingthe user defined action).

Moreover, in some implementations, the presentation of a notificationmay be provided to a device other than user device 106. For example, thepresentation may be provided to a device that is determined to be closeto the user or a device that the user will see or is looking at. Forexample, if the user device 106 of the user is not currently visible tothe user, but the user is viewing another device, the action triggerprocessing system 120 may determine to present the notification to thedevice the user is viewing.

Action trigger prompts can also be provided for many other types ofcontent that have varying parameter values. For example, an actiontrigger prompt may be provided for an advertisement. Selection of theprompt may enable the user to select various actions and events relatedto the product or service being advertised. By way of example, for anadvertisement advertising a product that is yet to be released, the usermay set a notification of the release that triggers on the release ofthe product. Conversely, for a product that is currently for sale, theuser may set an action to purchase the product when the product goes onsale, e.g., 10% off, by a particular retailer. In this latter example,the user may specify a particular sale price as the trigger, or the usermay specify any sale event as a trigger.

In situations in which the systems discussed herein collect personalinformation about users, or may make use of personal information, theusers may be provided with an opportunity to control whether programs orfeatures collect user information (e.g., information about a user'ssocial network, social actions or activities, profession, a user'spreferences, a user's current location, location trajectory, inferredlocations such as home/work, inferred context, calendar data, upcomingevents/bookings from email, etc.), or to control whether and/or how toreceive content from the content server that may be more relevant to theuser. In addition, certain data may be treated in one or more waysbefore it is stored or used, so that personally identifiable informationis removed. For example, a user's identity may be treated so that nopersonally identifiable information can be determined for the user, or auser's geographic location may be generalized where location informationis obtained (such as to a city, ZIP code, or state level), so that aparticular location of a user cannot be determined. Thus, the user mayhave control over how information is collected about the user and usedby a content server.

FIG. 5 is a block diagram of example mobile computing device. In thisillustration, the mobile computing device 510 is depicted as a handheldmobile telephone (e.g., a smartphone, or an application telephone) thatincludes a touchscreen display device 512 for presenting content to auser of the mobile computing device 510 and receiving touch-based userinputs. Other visual, tactile, and auditory output components may alsobe provided (e.g., LED lights, a vibrating mechanism for tactile output,or a speaker for providing tonal, voice-generated, or recorded output),as may various different input components.

The mobile computing device 510 may include mechanical or touchsensitive buttons 518 a-d. Additionally, the mobile computing device mayinclude buttons for adjusting volume output by the one or more speakers520, and a button for turning the mobile computing device on or off. Amicrophone 522 allows the mobile computing device 510 to convert audiblesounds into an electrical signal that may be digitally encoded andstored in computer-readable memory, or transmitted to another computingdevice. The mobile computing device 510 may also include a digitalcompass, an accelerometer, proximity sensors, and ambient light sensors.

The mobile computing device 510 may present a graphical user interfacewith the touchscreen 512. A graphical user interface is a collection ofone or more graphical interface elements and may be static (e.g., thedisplay appears to remain the same over a period of time), or may bedynamic (e.g., the graphical user interface includes graphical interfaceelements that animate without user input).

The mobile computing device 510 may include other applications,computing subsystems, and hardware. A voice recognition service 572 mayreceive voice communication data received by the mobile computingdevice's microphone 522, and translate the voice communication intocorresponding textual data or perform voice recognition.

The mobile computing device 510 may communicate wirelessly with one ormore networks to provide a variety of services, such as voice and dataservices.

Embodiments of the subject matter and the operations described in thisspecification can be implemented in digital electronic circuitry, or incomputer software, firmware, or hardware, including the structuresdisclosed in this specification and their structural equivalents, or incombinations of one or more of them. Embodiments of the subject matterdescribed in this specification can be implemented as one or morecomputer programs, i.e., one or more modules of computer programinstructions, encoded on computer storage medium for execution by, or tocontrol the operation of, data processing apparatus.

A computer storage medium can be, or be included in, a computer-readablestorage device, a computer-readable storage substrate, a random orserial access memory array or device, or a combination of one or more ofthem. Moreover, while a computer storage medium is not a propagatedsignal, a computer storage medium can be a source or destination ofcomputer program instructions encoded in an artificially generatedpropagated signal. The computer storage medium can also be, or beincluded in, one or more separate physical components or media (e.g.,multiple CDs, disks, or other storage devices).

The operations described in this specification can be implemented asoperations performed by a data processing apparatus on data stored onone or more computer-readable storage devices or received from othersources. The term “data processing apparatus” encompasses all kinds ofapparatus, devices, and machines for processing data, including by wayof example a programmable processor, a computer, a system on a chip, ormultiple ones, or combinations, of the foregoing. The apparatus caninclude special purpose logic circuitry, e.g., an FPGA (fieldprogrammable gate array) or an ASIC (application specific integratedcircuit). The apparatus can also include, in addition to hardware, codethat creates an execution environment for the computer program inquestion, e.g., code that constitutes processor firmware, a protocolstack, a database management system, an operating system, across-platform runtime environment, a virtual machine, or a combinationof one or more of them. The apparatus and execution environment canrealize various different computing model infrastructures, such as webservices, distributed computing and grid computing infrastructures.

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read only memory ora random access memory or both. The essential elements of a computer area processor for performing actions in accordance with instructions andone or more memory devices for storing instructions and data. Generally,a computer will also include, or be operatively coupled to receive datafrom or transfer data to, or both, one or more mass storage devices forstoring data, e.g., magnetic, magneto optical disks, or optical disks.However, a computer need not have such devices.

Embodiments of the subject matter described in this specification can beimplemented in a computing system that includes a back end component,e.g., as a data server, or that includes a middleware component, e.g.,an application server, or that includes a front end component, e.g., auser computer having a graphical user interface or a Web browser throughwhich a user can interact with an implementation of the subject matterdescribed in this specification, or any combination of one or more suchback end, middleware, or front end components. The components of thesystem can be interconnected by any form or medium of digital datacommunication, e.g., a communication network. Examples of communicationnetworks include a local area network (“LAN”) and a wide area network(“WAN”), an inter-network (e.g., the Internet), and peer-to-peernetworks (e.g., ad hoc peer-to-peer networks).

While this specification contains many specific implementation details,these should not be construed as limitations on the scope of anyinventions or of what may be claimed, but rather as descriptions offeatures specific to particular embodiments of particular inventions.Certain features that are described in this specification in the contextof separate embodiments can also be implemented in combination in asingle embodiment. Conversely, various features that are described inthe context of a single embodiment can also be implemented in multipleembodiments separately or in any suitable subcombination. Moreover,although features may be described above as acting in certaincombinations and even initially claimed as such, one or more featuresfrom a claimed combination can in some cases be excised from thecombination, and the claimed combination may be directed to asubcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. In certain circumstances, multitasking and parallel processingmay be advantageous. Moreover, the separation of various systemcomponents in the embodiments described above should not be understoodas requiring such separation in all embodiments, and it should beunderstood that the described program components and systems cangenerally be integrated together in a single software product orpackaged into multiple software products.

Thus, particular embodiments of the subject matter have been described.Other embodiments are within the scope of the following claims. In somecases, the actions recited in the claims can be performed in a differentorder and still achieve desirable results. In addition, the processesdepicted in the accompanying figures do not necessarily require theparticular order shown, or sequential order, to achieve desirableresults. In certain implementations, multitasking and parallelprocessing may be advantageous.

What is claimed is:
 1. A method, comprising: receiving, at a computerdevice, first instructions that cause a user device to render a resourcethat includes a first parameter value of a first parameter and displaythe resource; processing, by the computer device, the first instructionsto identify the first parameter value and determine whether the firstparameter value matches one of a plurality of predefined types; inresponse to determining that the first parameter value matches one ofthe plurality of predefined types: generating second instructions thatcause the user device to display an action trigger prompt that isrendered with the resource and in an initial state, and wherein theaction trigger prompt is rendered to augment the resource withadditional content not provided by rendering of the resource accordingto the first instructions, and the user device displays an actiontrigger menu in response to a selection of the action trigger prompt bya user, and in response to a selection of the action trigger prompt atthe user device, the second instructions further cause the user deviceto display an action trigger menu that includes an input for a userdefined event, receive an input of a first parameter of the user definedevent and a value for the first parameter in the action trigger menu;and causing, by the computer device, the second instructions to beprocessed with the first instructions by the user device.
 2. The methodof claim 1, wherein processing the first instructions comprisesprocessing a document object model of the resource to identify one ormore predetermined parameter types and values.
 3. The method of claim 1,wherein processing the first instructions comprises processing an imageincluded in the resource to identify textual content depicted in theimage and determining whether the textual content includes a firstparameter value that matches one of the plurality of predefinedparameter types.
 4. The method of claim 3, wherein processing the imagein the first resource further comprises processing the image todetermine location data of the textual content depicted in the imagethat specifies a location of the textual content.
 5. The method of claim4, wherein generating second instructions further comprises generatinginstructions that cause the user device to render the action triggerprompt at the location specified by the location data.
 6. The method ofclaim 1, wherein the computer device is the user device.
 7. The methodof claim 1, wherein the computer device is a computer that is separatefrom the user device.
 8. A system, comprising: a processor; and anon-transitory computer-readable medium coupled to the processor andhaving instructions stored thereon, which, when executed by theprocessor, cause the system to perform operations comprising: receiving,at the system, first instructions that cause a user device to render aresource that includes a first parameter value of a first parameter anddisplay the resource; processing, by the system, the first instructionsto identify the first parameter value and determine whether the firstparameter value matches one of a plurality of predefined types; inresponse to determining that the first parameter value matches one ofthe plurality of predefined types: generating second instructions thatcause the user device to display an action trigger prompt that isrendered with the resource and in an initial state, and wherein theaction trigger prompt is rendered to augment the resource withadditional content not provided by rendering of the resource accordingto the first instructions, and the user device displays an actiontrigger menu in response to a selection of the action trigger prompt bya user, and in response to a selection of the action trigger prompt atthe user device, the second instructions further cause the user deviceto display an action trigger menu that includes an input for a userdefined event, receive an input of a first parameter of the user definedevent and a value for the first parameter in the action trigger menu;and causing, by the system, the second instructions to be processed withthe first instructions by the user device.
 9. The system of claim 8,wherein processing the first instructions comprises processing adocument object model of the resource to identify one or morepredetermined parameter types and values.
 10. The system of claim 8,wherein processing the first instructions comprises processing an imageincluded in the resource to identify textual content depicted in theimage and determining whether the textual content includes a firstparameter value that matches one of the plurality of predefinedparameter types.
 11. The system of claim 10, wherein processing theimage in the first resource further comprises processing the image todetermine location data of the textual content depicted in the imagethat specifies a location of the textual content.
 12. The system ofclaim 11, wherein generating second instructions further comprisesgenerating instructions that cause the user device to render the actiontrigger prompt at the location specified by the location data.
 13. Thesystem of claim 8, wherein the system is the user device.
 14. The systemof claim 8, wherein system is a computer that is separate from the userdevice.
 15. A non-transitory computer-readable medium havinginstructions stored thereon, which, when executed by a processor in acomputer apparatus, cause the computer apparatus to perform operations,comprising: receiving first instructions that cause a user device torender a resource that includes a first parameter value of a firstparameter and display the resource; processing the first instructions toidentify the first parameter value and determine whether the firstparameter value matches one of a plurality of predefined types; inresponse to determining that the first parameter value matches one ofthe plurality of predefined types: generating second instructions thatcause the user device to display an action trigger prompt that isrendered with the resource and in an initial state, and wherein theaction trigger prompt is rendered to augment the resource withadditional content not provided by rendering of the resource accordingto the first instructions, and the user device displays an actiontrigger menu in response to a selection of the action trigger prompt bya user, and in response to a selection of the action trigger prompt atthe user device, the second instructions further cause the user deviceto display an action trigger menu that includes an input for a userdefined event, receive an input of a first parameter of the user definedevent and a value for the first parameter in the action trigger menu;and causing the second instructions to be processed with the firstinstructions by the user device.